Multimodal Fusion for Cognitive Load Measurement in an Adaptive Virtual Reality Driving Task for Autism Intervention

A virtual reality driving system was designed to improve driving skills in individuals with autism spectrum disorder (ASD). An appropriate level of cognitive load during training can help improve a participant’s long-term performance. This paper studied cognitive load measurement with multimodal information fusion techniques. Features were extracted from peripheral physiological signals, Electroencephalogram (EEG) signals, eye gaze information and participants’ performance data. Multiple classification methods and features from different modalities were used to evaluate participant’s cognitive load. We verified classifications’ result with perceived tasks’ difficulty level, which induced different cognitive load. We fused multimodal information in three levels: feature level, decision level and hybrid level. The best accuracy for cognitive load measurement was 84.66%, which was achieved with the hybrid level fusion.